Emotional response research has been an area of active research for decades. Some of the research has used self-reporting techniques. Izard (Izard, C. E. “The emotions in life and science” In Human emotions, C. E. Izard, ed., New York, Plenum Press, 1977, pp. 1-18) and Plutchik (Plutchik, R. “Emotions: A general psychoevolutionary theory” In Approaches to emotion, K. R. Scherer, and P. Ekman, eds., Hillsdale, N.J., Lawrence Erlbaum Associates, 1984, pp. 197-219) both used a discrete self-reporting approach that focused on specific emotions such as happiness and anger. Others have used a more robust three-dimensional self-report approach (Osgood, C. E. et al. “The measurement of meaning” Urbana, Ill., University of Illinois Press, 1958; Russell, J. A. and Mehrabian, A. Journal of Research in Personality, 1977, 11:273-294; Sundar, S. S., and Kalyanaraman, S. Journal of Advertising, 2004, 33:7-17). Other research has measured emotional response using physiological measures. Agreement between self-reporting and physiological measurements would provide convergent validity for both methods. However, to date, a link between the two measures of emotion has not been fully explored.
The search for physiological links to emotion reflects an approach that seeks fundamental or discrete emotions (Mandler, G. “Mind and body” New York, Norton, 1984). Subcortical emotional responses are amply recorded through classical conditioning of fear reactions to audio or visual stimuli (LeDoux, J. E. et al. Journal of Cognitive Neuroscience, 1989, 1:238-243). The responses either interrupt the cognitive focus of current attention or influence the context for ongoing cognitive processes (Simon, H. A. Comments. In Affect and cognition: The seventeenth annual Carnegie symposium on cognition, M. S. Clarke, and S. T. Fiske, eds., Hillsdale, N.J., Erlbaum, 1982, pp. 333-342). Pleasant and unpleasant emotional responses were found to increase neural activity in the medial prefrontal cortex, thalamus, and hypothalamus, while unpleasant emotions were found to increase the neural activity in the occipitotemporal cortex, parahippocampal gyms, and amygdale (Lane, R. D. et al. Neuropsychologia, 1997, 35:1437-1444). Additionally, facial expressions of disgust or anger were found to increase the neural activity in the left inferior frontal gyrus (Sprengelmeyer, R. et al. Proceedings of the Royal Society of London, Series B: Biological Sciences, 1998, 265:1927-1931). A meta-analysis of emotion activation studies in PET and fMRI (Phan, K. L. et al. NeuroImage, 2002, 16:331-348) concluded that no single brain region is activated by all emotions, and no single brain region is activated by one particular emotion.
The discrete approach assumes categorical judgment of emotional stimuli. This requires connections between both hemisphere (Bowers, D. et al. Brain, 1991, 114:2593-2609) and between the anterior cingulated cortex and the bilateral prefrontal cortex (Devinsky, O. et al. Brain, 1995, 118:279-306). Therefore, certain brain regions might be activated because of the demand to categorize or label discrete emotions rather than the natural emotional responses to given stimuli. Furthermore, most neuroimaging studies treat emotions as two discrete categories—pleasant and unpleasant—while ignoring the nuances along the pleasure dimension and the additional explanatory power of the arousal and dominance dimensions. For example, intensity of fear has been associated with brain activities in the left inferior frontal gyrus (Morris, J. S. et al. Brain, 1998, 121:47-57) while anger and disgust have been associated with difference degrees of intensity or arousal (Iidaka, T. et al. Journal of Cognitive Neuroscience, 2001, 13:1035-1047).
The alternative three-dimensional approach to emotion attempts to simplify the representation of responses by identifying a set of common dimensions that can be used to distinguish specific emotions from one another. This approach, which includes both verbal and non-verbal measures (Lang, P. J. Behavioral treatment and bio-behavioral assessment: Computer applications. In Technology in mental health care delivery systems, J. B. Sidowski, J. H. Johnson, and T. A. Williams, eds., Norwood, N.J., Ablex, 1980, pp. 119-137; Osgood, C. E. et al. The measurement of meaning, Urbana, Ill., University of Illinois Press, 1958; Russell, J. A., and Mehrabian, A. Journal of Research in Personality, 1977, 11:273-294), has been largely ignored in research.
One example of the approach is a pleasure-displeasure, arousal-calm, and dominance-submissiveness (PAD) model (Russell, J. A., and Mehrabian, A. Journal of Research in Personality, 1977, 11:273-294). The three bi-polar dimensions are independent of each other, and the variance of emotional responses can be well identified with their positions along the three dimensions. The dimensional approach helps differentiate emotions posited by the discrete approach (Shaver, P. et al. Journal of Personality and Social Psychology, 1987, 52:1061-1086) by providing a numeric level of each dimension to describe the specific emotions. Each discrete emotion can be identified by specific combinations of the dimensions. The meaning of these specific adjectives may differ by individual, culture or some other influence; however, the method for identifying the response is universal. Understanding the reactions by dimensions, as first proposed by Osgood, Suci, and Tannenbaum (Osgood, C. E. et al. The measurement of meaning, Urbana, Ill., University of Illinois Press, 1958), makes the analysis methodologically meaningful.
Neuroimaging techniques, such as positron emission tomography (PET) and functional magnetic resonance imaging (fMRI), are reported in a growing body of emotion literature. Regional cerebral blood flow (rCBF) signals of PET and blood oxygenation level-dependent (BOLD) signals of fMRI are used to identify possible links between brain regions and emotions. It is worthwhile to point out that the search for these links reflects a Darwinian approach to emotions, which seeks evolutionarily fundamental or discrete emotions (Mandler, G. (1984) Mind and body. New York: Norton; Plutchik, R. (1980) “A general psychoevolutionary theory of emotion” In R. Plutchik & H. Kellerman (Eds.), Emotion: Theory, research, and experience, Vol. 1, pp. 3-33, New York: Academic), and the alternative three-dimensional approach with verbal or non-verbal measures (Lang, P. J. (1980) “Behavioral treatment and bio-behavioral assessment: Computer applications” In J. B. Sidowski, J. H. Johnson & T. A. Williams (Eds.), Technology in mental health care delivery systems (pp. 119-137), Norwood, N.J.: Ablex; Morris, J. D. et al. (1993) “Assessing emotional responses to advertisements with (SAM) the self-assessment manikin” In Proceedings of the Allied Southern Business Association 1993 Annual Meeting: Allied Southern Business Association; Morris, J. D. et al. (2002) Journal of Advertising Research, 42(3):7-17; Osgood, C. E. et al. (1958), The measurement of meaning. Urbana, Ill.: University of Illinois Press; Russell, J. A. and Mehrabian, A. (1977) Journal of Research in Personality, 11:273-294) is largely ignored.
With the discrete approach to emotions, researchers have used visual, audio, and video stimuli and reported mixed findings about the links in question. For example, when visual stimuli such as facial emotional expressions were used, both pleasant and unpleasant emotions were found to increase brain activities in the medial prefrontal cortex, thalamus, hypothalamus, whereas unpleasant emotions were found to increase the activities in the occipitotemporal cortex, parahippocampal gyms, and amygdale (Lane, R. D. et al. (1997) Neuropsychologia, 35(11):1437-1444). Additionally, facial expressions of disgust or anger were found to increase the activities in the left inferior frontal gyms (Sprengelmeyer, R. et al. (1998) Proceedings of the Royal Society of London, Series B: Biological Sciences, 265:1927-1931). When audio stimuli such as speech were used, emotional responses to the auditory information of a speech, such as intonation, loudness, and tempo, were found to increase brain activities in the inferior frontal gyms whereas emotional responses to the semantic information of the speech were found to increase brain activities in the middle frontal gyms (George, M. S. et al. (1996) Archives of Neurology, 53:665-670). Additionally, the middle temporal gyms was found to integrate the processing of both the auditory and the semantic information of the speech as well as information from past memories and interpret the meaning of the speech (Mitchell, R. L. C. et al. (2003) Neuropsychologia, 41:1410-1421; Ojemann, G. A. and Schoenfield-McNeill, J. (1998) Brain and Language, 64:317-327). When both audio and visual stimuli such as video clips were used, fear, disgust, and sadness were found to increase brain activities in orbitofrontal cortex, medial prefrontal cortex, and thalamus, whereas happiness was found to increase the activities in the entorhinal cortex, the medial prefrontal cortex, and thalamus (Lane, R. D. et al. (1997) American Journal of Psychiatry, 154:926-933; Paradiso, S. et al. (1997) American Journal of Psychiatry, 154:384-389). Moreover, the semantic information of video clips was found to increase brain activities in the inferior frontal gyrus and the very use of dynamic stimuli such as video clips is more likely to induce brain activities in the inferior frontal gyrus than are still face images (Adolphs, R. (2002) Behavioral and Cognitive Neuroscience Reviews, 1:21-62; Decety, J. and Chaminade, T. (2003) Neuropsychologia, 41:127-138).
As summarized in a meta-analysis of emotion activation studies in PET and fMRI (Phan, K. L. et al. (2002) NeuroImage, 16:331-348), no single brain region is activated by all emotions, nor is a single brain region activated by one specific emotion. Nevertheless, from a cortical emotional learning perspective, three brain regions, namely the amygdala, prefrontal cortex, and temporal cortex, are particularly important, especially for video clips. First, the prefrontal cortex has been found to have neural projections to the amygdale (McDonald, A. J. et al. (1996) Neuroscience, 71:55-75), and the temporal cortex has been found to send stimulus information to the prefrontal cortex (Hasselmo, M. E. et al. (1989) Behavioral Brain Research, 32:203-218). Second, the temporal cortex has been found to allow the brain to merge perceptual and semantic information, past memories and short-term manipulation of the stimuli (Mitchell, R. L. C. et al. (2003) Neuropsychologia, 41:1410-1421). Finally, both the semantic information and the dynamic nature of video clips have been found to increase brain activities in the prefrontal cortex (Adolphs, R. (2002) Behavioral and Cognitive Neuroscience Reviews, 1:21-62; Decety, J. and Chaminade, T. (2003) Neuropsychologia, 41:127-138).
As mentioned above, the discrete approach to emotions dominates the current neuroimaging studies. However, there are conceptual problems with this approach and researchers would propose different types of discrete emotions. For example, they could be a set of eight emotions—interest, surprise, joy, anguish, fear, shame, disgust, and rage (Tomkins, S. S. (1980) “Affect as amplification: Some modifications in theory” In R. Plutchik & H. Kellerman (Eds.), Emotion: Theory, research and experience. New York: Academic Press), or another set of eight emotions—fear, anger, joy, sadness, acceptance, disgust, anticipation, and surprise (Plutchik, R. (1984) “A general psychoevolutionary theory” In K. R. Scherer & P. Ekman (Eds.), Approaches to emotion, Hillsdale, N.J.: Lawrence, Erlbaum Associates), or even another set of ten emotions—interest, joy, surprise, distress, anger, disgust, contempt, shame, fear, and guilt (Izard, C. E. (1977) “The emotions in life and science” In C. E. Izard (Ed.), Human emotions (pp. 1-18), New York: Plenum Press). The disagreement on either the number or the kinds of emotions across the three lists seems to suggest a lack of well-accepted theoretical ground to support the notion of discrete emotions (Mandler, G. (1984). Mind and body. New York: Norton).
Alternatively, the present inventors have found that the dimensional approach to emotions exists in the brain. This approach will simplify the representation of emotional responses by identifying a set of common dimensions that can be used to distinguish specific emotions from one another. Two of three bi-polar independent dimensions have been identified in discrete locations in the brain. Levels of pleasure-displeasure, arousal-calm, (PA) (Russell, J. A. and Mehrabian, A. (1977) Journal of Research in Personality, 11:273-294) have been measured and located. As a matter of fact, the dimensional approach, such as the PA, helps differentiate separate emotions posited by the discrete approach by combining levels of pleasure and arousal (Shaver, P. et al. (1987) Journal of Personality and Social Psychology, 52(6):1061-1086).
Brain activity identified using the dimensional approach provides a very promising new perspective for investigation, specifically measuring issues ignored and under-explored in previous neuroimaging studies. For example, categorical judgment of emotional stimuli requires connections between both hemispheres (Bowers, D. et al. (1991) Brain, 114:2593-2609) and between the anterior cingulated cortex and the bilateral prefrontal cortex (Devinsky, O. et al. (1995) Brain, 118:279-306); therefore, certain brain regions may be activated by natural reactions to given stimuli, as levels of pleasure, arousal and dominance rather than discrete emotions. Furthermore, most neuroimaging studies treat emotions as falling into two large categories of pleasant or unpleasant and ignore the nuance along the pleasure dimension and the additional explanatory power of the arousal and dominance dimensions. For instance, it has been found that brain activities in the left inferior frontal gyms would be enhanced when the intensity of fearful facial expression is increased (Morris, J. S. et al. (1998) Brain, 121:47-57) and anger and disgust have varying degree of intensity (Iidaka, T. et al. (2001) Journal of Cognitive Neuroscience, 13(8):1035-1047). The association between arousal, which well represents intensity, and brain regions can now be investigated in a systematic manner.
Another line of emotional response research focuses on somatovisceral reactions to emotional stimuli. These reactions are primarily autonomic nervous system (ANS) activities such as heart rate, sweating, and digestive processes (Zajonc, R. B. et al. (1993) “Brain temperature and subjective emotional experience” In M. Lewis & J. M. Haviland (Eds.), Handbook of emotions. New York: Guilford Press). Similar to the dominant approach in neuroimaging studies, researchers again attempted to identify ANS activities of discrete emotions. Unfortunately, this attempt is not very successful. For example, the ANS activity may vary as a function of the intensity of emotions rather than specific emotions per se. When the influence of specific emotions on ANS activities is weak or the influence in question is not strictly additive, changes in ANS activities are largely due to non-emotional factors such as individual differences (Cacioppo, J. T. et al. (1992) Journal of Personality and Social Psychology, 62:110-128; Stemmler, D. G. (1992) Differential psychophysiology: Persons in situations. New York: Springer-Verlag), anticipated or actual somatic activities (Obrist, P. A. et al. (1970) Psychophysiology, 6:569-587), respiration (Bernston, G. C. et al. (1993) Psychophysiology, 30:183-196), and attention (Graham, F. K. and Clifton, R. K. (1966) Psychological Bulletin, 65:305-320; Lacey, J. I. et al. (1963) “The visceral level: Situational determinants and behavioral correlates of autonomic response patterns” In P. H. Kapp (Ed.), Expression of the emotions in man (pp. 161-196), New York: International Universities Press). The ANS activities may also be a function of the perception of specific emotions because anticipated or realized action requirements of an emotional challenge would determine physiological responses to the stimulus (Frijda, N. H. (1986). The emotions. New York: Cambridge University Press; Lang, P. J. et al. (1990) Psychological Review, 97:377-395). Because there are strong correlations between brain activities and ANS activities (Matthews, S. C. et al. (2004) NeuroImage, 22(3):1151-1156), it is not surprising to note the insufficiency of the discrete approach to emotions in defining emotion-specific ANS activities, which again leads to the present inventors' advocacy of the dimensional approach to emotions.
Methods for detecting emotional response to stimuli based on brain activity using functional brain imaging and other techniques have been developed. See, for example, U.S. Pat. No. 6,099,319 (Zaltman and Kosslyn, filed Nov. 9, 1998, “Neuroimaging as a Marketing Tool”), U.S. Pat. No. 6,292,688 (Patton, filed Feb. 28, 1996, “Method and Apparatus for Analyzing Neurological Response to Emotion-Inducing Stimuli”), and U.S. Patent Publication US 2005/0154290 A1 (Langleben, filed Jun. 17, 2002, “Functional Brain Imaging for Detecting and Assessing Deception and Concealed Recognition, and Cognitive/Emotional Response to Information”).
It would be advantageous to have available a method for assessing emotional response that incorporates two independent measures, both a psychological, non-verbal measure and a neural-physiological measure, in order to cross-validate and to correct for scale bias or measurement errors associated with single measure approaches.